Computational procedures for optimal stopping problems for Markov chains
نویسندگان
چکیده
منابع مشابه
Optimal Stopping Problems for Some Markov Processes
In this paper, we solve explicitly the optimal stopping problem with random discounting and an additive functional as cost of observations for a regular linear diffusion. We also extend the results to the class of one-sided regular Feller processes. This generalizes the result of Beibel and Lerche [Statist. Sinica 7 (1997) 93–108] and [Teor. Veroyatn. Primen. 45 (2000) 657–669] and Irles and Pa...
متن کاملOptimal Starting-Stopping Problems for Markov-Feller Processes
By means of nested inequalities in semigroup form we give a characterization of the value functions of the starting-stopping problem for general Markov-Feller processes. Next, we consider two versions of constrained problems on the final state or on the final time. The plan is as follows:
متن کاملEecient Stopping Rules for Markov Chains
Let M be the transition matrix, and the initial state distribution , for a discrete-time nite-state irreducible Markov chain. A stopping rule for M is an algorithm which observes the progress of the chain and then stops it at some random time ?; the distribution of the nal state is denoted by ?. We give a useful characterization for stopping rules which are optimal for given target distribution...
متن کاملStopping rule reversal for finite Markov chains
Consider a finite irreducible Markov chain with transition matrix M = (pij). Fixing a target distribution τ , we study a family of optimal stopping rules from the singleton distributions to τ . We show that this family of rules is dual to a family of (not necessarily optimal) rules on the reverse chain from the singleton distributions to a related distribution α̂ called the τ -contrast distribut...
متن کاملReachability problems for Markov chains
We consider the following decision problem: given a finite Markov chain with distinguished source and target states, and given a rational number r, does there exist an integer n such that the probability to reach the target from the source in n steps is r? This problem, which is not known to be decidable, lies at the heart of many model checking questions on Markov chains. We provide evidence o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Mathematical Analysis and Applications
سال: 1969
ISSN: 0022-247X
DOI: 10.1016/0022-247x(69)90259-5